We develop a practical approach to establish the stability, that is, therecurrence in a given set, of a large class of controlled Markov chains. Theseprocesses arise in various areas of applied science and encompass importantnumerical methods. We show in particular how individual Lyapunov functions andassociated drift conditions for the parametrized family of Markov transitionprobabilities and the parameter update can be combined to form Lyapunovfunctions for the joint process, leading to the proof of the desired stabilityproperty. Of particular interest is the fact that the approach applies even insituations where the two components of the process present a time-scaleseparation, which is a crucial feature of practical situations. We then move onto show how such a recurrence property can be used in the context of stochasticapproximation in order to prove the convergence of the parameter sequence,including in the situation where the so-called stepsize is adaptively tuned. Wefinally show that the results apply to various algorithms of interest incomputational statistics and cognate areas.
展开▼
